AD-A091  600  CITY  COLL  NEW  YORK  DEPT  OF  COMPUTER  SCIENCE  F/G  12/1 

STEEPEST  EDGE  ALGORITHMS  IN  LINEAR  AND  NONLINEAR  PROGRAMMING. (U) 

SEP  80  0  GOLDFARB  DAAG29-77-G-0114 

UNCLASSIFIED  80-1  AR0-14180.3-M  NL 


AD  A0916°8 


U.S.  ARMY  RESEARCH  OFFICE 

^7 - - 

GRANT  NUMBER  pDAAqfg9-77-G-^ll4  J 


t'  i-S  ■  -  y,  , 
JOtni  7/-3J-  f"  \ 


CP 

¥ke  City  Colle 


-^^New  York , 


6^ 


,  - r>  \  Depa»rfcM<mt,  of  Computer  Science  .  J  KU 

K - *  Hw  Yerk")  M.  Y.  10071 

DTIC 


SELECTED 
NOV  1  2 1980 1  ■ 


_  UNCLASSIFIED _ 

SECURITY  CLASSIFICATION  OF  THIS  PAGE  (Whan  O M  Cntotod) 


REPORT  DOCUMENTATION  PAGE 


RERORT  NUMBER 


B  A/*p  READ  INSTRUCTIONS 

_ BEFORE  COMPLETING  FORM 

2.  GOVT  ACCESSION  NO.  J  RECIPIENT'S  CAT ALOG  NUMBER 


_ 80-1 _ ^ _ M-  * 

«•  title  (m>d  subiuia)  Steepest  Edge  Algorithms  for 
Linear  and  Nonlinear  Prograjanfing 


CESSION  NO.  i.  RECIPjEN 

if  06 


1 7.  AUTHORfaJ 


DONALD  GOLDFARB 


PERFORMING  ORGANIZATION  NAME  AND  AOORESS 


5.  TYPE  OF  REPORT  ft  PERIOD  COVERED 

31  July  1980 

6.  PERFORMING  ORG.  REPORT  NUMBER 
ft.  CONTRACT  OR  GRANT  NUMBErT^ 

Grant 

10.  PROGRAM  ELEMENT.  PROJECT.  TASK 
AREA  «  WORK  UNIT  NUMBERS 


The  City  College  of  the  City  University  of  New  Y 
Dept,  of  Computer  Science  ' 

- K»w  York,  ■  N.  Y„  10033 - 

n.  CONTROLLING  OFFICE  NAME  ANO  AOORESS 

U.  S.  Army  Research  Office 

Post  Office  Box  12211 

Research  Triangle  Park,  NC  27709 

U.  MONITORING  AGENCY  NAME  a  AOORESVtf  dl llatanl  trooi  Controlling  Olllea) 

Dept,  of  the  Navy 

Office  of  Naval  Research  Resident  Representative 
715  Broadway  (5th  Floor) 

Ww  York,  N.  Y.  1QQQ3 - 

IB.  DISTRIBUTION  STATEMENT  (ol  thla  Raport) 


12.  REPORT  DATE 

September,  1980 

IS.  NUMBER  OF  RAGES 


IS.  SECURITY  CLASS,  (ol  thla  raport) 


Unclassified 

1S«.  OECL  ASSIFICATION/  DOWNGRADING 
SCHEDULE 


Approved  for  public  release;  distribution  unlimited. 


[97.  DISTRIBUTION  STATEMENT  (ot  th»  mbatrmct  m torod  In  Block  20,  If  dlltoront  from  Ropori) 


[  ift.  supplementary  notes 


The  view,  opinions,  and/or  findings  contained  in  this  report  are  those  of  the 
author (s)  and  should  not  be  construed  as  an  official  Department  of  the  Army 
position,  policy,  or  decision,  unless  so  designated  by  other  documentation. 

If.  Key  WORDS  (Conilnuo  on  .......  mldo  II  noeooaory  and  taonttty  Or  otocm  numotr)  ^ 

Optimization ,  Linear  iTcgramming,  Quadratic  Programming,  Nonlinear  Programming, 
Steepest  Edge  Algorithm,  Network  Flows,  Maximum  Flow,  Ellipsoid  Method,  Compu¬ 
tational  Complexity,  Steplength  Algorithms,  Negative  Curvature,  LU  factorization 
Sparse  Matrices 

lft»  ABSTRACT  fCMftM  «■  rowoeom  af*  It  nmoooomry  Idonilty  by  block  numbor) 

^  Several  algorithms  in  linear  and  nonlinear  programming  have  been  developed 

and  analyzed.  A  worst-case  analysis  of  the  steepest  edge  simplex  method  showed 
it  to  have  exponential  time-complexity.  This  algorithm  was  specialized  for 
solving  minimum  cost  network  flow  problems  and  a  partial  pricing  variant  was 
developed.  After  extensive  testing  on  randomly  generated  large  problems 
the  method  was  found  to  be  inferior  to  efficiently  coded  partial  pricing - 


00  ,'STn  1473  EDITION  or  •  MOV  •>  IS  OBSOLETE 


UNCLASSIFIED 


Block  20 

variants  of  the  "standard"  simplex  method  except  for  the  max  flow  problem  and  the 
problem  of  finding  a  feasible  flow.  On  the  max  flow  problem  the  algorithm  was  compared 
with  the  Edmonds-Karp  and  Dinic  algorithms  and  found  to  be  superior.  In  quadratic 
programming,  the  use  of  the  steepest  edge  (face)  criterion  for  dropping  constraints 
was  found  to  be  helpful.  Dual  methods  were  found  to  be  even  more  efficient 
and  their  use  in  recursive  quadratic  programming  algorithms  for  nonlinear 
programming  problems  is  recommended.  A  numerically  stable  ellipsoid  algorithm 
for  linear  programming  was  developed  and  analyzed.  At  present  the  main  promise 
that  this  method  holds  is  as  a  powerful  theoretical  tool.  Several  minimization 
algorithms  for  taking  advantage  of  negative  curvature  were  developed  as  was  a 
curvilinear  steplength  algorithm  which  ensures  convergence  to  a  positive  semidefinite 
point. 
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A.  PROBLEM  STATEMENT 


Although  we  originally  only  proposed  to  study  steepest  edge  algorithms 
in  linear  and  nonlinear  programming,  exciting  new  developments  and 
and  our  own  scientific  curioudity  led  us  to  broaden  our  research  into 
other  types  of  algorithms  in  linear  and  nonlinear  programming  as  well. 

Our  researches,  however,  remained  limited  to  a  study  of  algorithms 
in  these  two  areas. 

In  linear  programming  a  number  of  problems  were  studied  including 

(i)  the  computational  complexity  of  the  steepest  edge  simplex  algorithm, 

(ii)  the  ellipsoid  algorithm  of  Shor  and  Judin  and -Nemirovskii,  which 
was  used  by  Wiachiyan  to  show  that  linear  programs  could  be  solved 

in  polynomial  time,  and  (iii)  the  preservation  of  sparsity  when  up¬ 
dating  the  LU  factorization  of  the  basis  matrix  in  the  simplex  method. 

For  the  subclass  of  linear  programming  problems  which  are  subsumed 
under  the  category  of  minimum  cost  network  flow  problems,  we 

studied  (i)  the  development  of  efficient  steepest  edge  simplex  algorithms 
for  these  problems,  (ii)  how  to  choose  appropriate  data  structures 
to  implement  these  algorithms,  (iii)  the  development  of  specialized 
algorithms  for  the  max  flow  problem  and  (iv)  the  avoidance  of  de¬ 
generate  pivots. 

In  the  closely  related  subject  of  quadratic  programming  we  studied 
(i)  steepest  edge  (i.e.,  face)  algorithms,  and  (ii)  dual  and  primal- 
dual  algorithms. 

In  the  area  of  nonlinear  programming  we  studied  the  problems  of  (i) 
developing  algorithms  which  can  take  advantage  of  negative  curvature 
in  the  objective  function,  and  (ii)  how  to  determine  the  steplength 
to  take  along  a  curvilinear  path  in  a  region  where  the  function 
being  minimized  is  nonconvex. 

B.  SUMMARY  OF  RESEARCH  RESULTS 

1.  Linear  Programming  Algorithms 

(H  Computational  Complexity  of  the  Steepest  Edge  Simplex  Algoithm. 

We  have  investigated  the  computational  complexity  of  the  steepest- 
edge  simplex  algorithm  for  general  linear  programming  problems. 

We  have  shown  how  to  construct  linear  programs  whose  feasible 
regions  are  combinatorially  equivalent  to  hypercubes  of  arbitrary 
dimension  n  for  which  the  steepest  edge  simplex  algorithm  passes 
through  all  2  vertices.  This  "worst-case"  analysis  extends  the 
well-known  analogous  result  of  Klee  and  Minty  for  the  standard 
pivot  section  rule. 


(ii)  The  Ellipsoid  Method 

A  rigorous  proof  has  been  given  of  the  fact  that  among  all  ellipsoids 
that  contain  a  given  half-ellipsoid  the  one  constructed  by  the 
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ellipsoid  method  is  of  minimum  volume.  The  deep-cut  version  of 
the  ellipsoid  method  was  analyzed  for  a  special  class  of  problems 
and  it  was  shown  that  this  version  is  only  four  to  five  times 
as  fast  as  the  Shor-Khachiyan  version.  It  was  also  demonstrated 
that  although  the  volumes  of  the  ellipsoids  generated  converge 
to  zero,  the  points  (i.e.  centers  of  the  ellipsoids)  need  not 
converge  to  a  feasible  point  if  the  feasible  set  is  not  full-dimensional 
Also,  in  the  infeasible  case  it  was  shown  that  infeasibility  can 
go  undetected. 

A  survey  of  past  and  current  work  on  the  ellipsoid  method  was 
written  in  collaboration  with  Profs.  Robert  Bland  and  Michael 
Todd  of  Cornell  University,  and  a  numerically  stable  algorithm 
was  developed  for  solving  linear  programming  problems  in 
collaboration  with  Professor  Michael  Todd.  Our  implementation 
updates  at  each  iteration  the  Cholesky  factors  of  the  symmetric 
positive  definite  matrix  which  defines  the  ellipsoids  in  a  way 
which  ensures  numerical  stability.  By  using  so-called  "surrogate" 
cuts  and  objective  function  cuts  and  by  refining  both  lower  and 
upper  bounds  on  the  optimal  value  of  the  objective  function,  our 
algorithm  should  be  able  to  terminate  with  an  indication  of  in¬ 
feasibility  or  with  a  provably  good  feasible  solution  in  a  moderate 
number  of  iterations.  The  method  was  coded  and  tested  on  a  small 
set  of  problems.  Unfortunately,  the  results  obtained  were  not 
very  promising  even  though  all  sorts  of  devices  to  accelerate 
convergence  were  employed. 

( iii )  Updating  of  Sparse  LU  Factorization  of  an  LP  Basis  Matrix: 

A  new  way  to  preserve  as  much  sparsity  as  possible  when  updating 
the  LU  factorization  of  a  basis  matrix  (in  the  simplex  method, 
for  example)  has  been  developed.  By  proper  choice  of  row  and 
column  permutations  it  produces  a  block  upper  triangular  matrix 
with  small  diagonal  blocks  which  is  then  triangularized  by  the 
Bartels-Golub  algorithm. 

Minimum  Cost  Network  Flow  Algorithms 

(i)  Steepest  Edge  Simplex  Algorithms. /(ii)  Implementation. 

A  steepest  edge  simplex  algorithm  for  network  flow  problems  has 
been  developed  which  updates  the  number  of  arcs  in  each  nonbasic 
cycle,  (i.e.,  flow  augmenting  path),  after  each  simplex  pivot 
from  properties  of  the  basic  tree.  A  partial  pricing  variant 
of  this  network  steepest  edge  simplex  method  has  also  been 
developed.  A  network  simplex  code  using  list  structures  for 
storing  the  basic  tree  and  including  several  partial  pricing 
strategies  as  options  has  been  written  in  FORTRAN. 


A  novel  feature  of  the  code  is  the  use  of  a  postorder  thread 
to  represent  the  basis  tree.  This  code  was  subjected  to  extensive 


-3- 


computational  testing  on  thirty  randomly  generated  large  problems 
(thousands  of  nodes  and  tens  of  thousands  of  arcs)  -  to  determine 
the  most  efficient  level  of  partial  pricing. 

Our  principal  finding  is  that  the  decrease  in  iterations  that 
results  from  using  the  steepest  edge  column  selection  rule  in  either 
a  full  or  partial  pricing  algorithm  does  not  compensate  sufficiently 
for  the  increase  in  work  per  iteration  over  that  required  by  the 
"minimum  reduced  cost"  rule  on  most  network  flow  problems.  Exceptions 
to  this  are  the  max  flow  problem  and  the  problem  of  finding  a  feasible 
flow. 

(iii)  Max  Flow  Algorithm: 

A  partial  pricing  variant  of  the  steepest  edge  simplex  algorithm  for 
minimum  cost  network  flow  problems  was  found  to  be  very  efficient 
for  solving  max  flow  problems.  A  FORTRAN  program  for  solving  max 
flow  problems  by  the  steepest  edge  simplex  method  was  developed 
in  collaboration  with  Dr.  Michael  Grigoriadis  on  the  basis  of 
his  RNET  code.  This  was  compared  computationally  with  our  own 
FORTRAN  codes  for  the  Edmonds-Karp  algorithm  and  Dinic's  algorithm. 

Our  results  indicate  that  our  method  is  superior  to  the  latter 
method  and  that  both  the  steepest  edge  and  Dinic  algorithms  are 
vastly  superior  to  the  Edmonds-Karp  algorithm. 

(iv)  Avoiding  Degenerate  Pivots. 

Two  techniques  for  avoiding  degenerate  pivots  in  network  flow 
problems  have  been  developed,  but  not  yet  computationally  tested. 

They  are  based  upon  the  use  of  special  list  structures. 

3.  Quadratic  Programming  Algorithms. 

IT)  Steepest  Edge  (Face)  Algorithms . 

The  steepest  edge (face) criterion  for  dropping  constraints  from 
an  active  set  in  primal  quadratic  programming  algorithms  has  been 
implemented  in  a  FORTRAN  code. 

Extensive  computational  testing  indicated  that  use  of  the  steepest 
face  criterion  results  in  fewer  steps  and  basis  changes.  However 
because  of  the  increased  computational  requirements  per  iteration, 
there  is  practically  no  change  in  the  total  work  required  when 
compared  with  algorithms  using  standard  column  selection  criteria. 

(ii)  Dual  and  Primal-Dual  Algorithms. 

A  numerically  stable  dual  method  for  positive  definite  quadratic 
programs  has  been  developed  and  fully  implemented  in  FORTRAN. 

It  uses  both  Cholesky  and  QR  factorizations.  Extensive  computational 
testing  indicates  that  it  is  three  to  four  times  as  efficient  as 
competitive  primal  codes.  When  used  as  a  subroutine  in  the  Harwell 
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code  of  Powell's  recursive  quadratic  programming  algorithm  for 
solving  nonlinear  programming  problems  a  similar  speed-up  in 
performance  was  observed. 

Nonlinear  Programming  Algorithms 

(  i)  Minimization  Algorithms  which  Make  Use  of  Negative  Curvature. 

Several  minimization  algorithms  which  make  use  of  negative  curvature 
have  been  developed.  These  include  versions  of  the  Goldfeld-Quandt- 
Trotter  method  and  gradient  path  methods  which  incorporate  negative 
curvature  information  through  the  numerically  stable  Bunch- 
Parlett  factorization. 

(ii)  Curvilinear  Path  Steplength  Algorithms 

A  curvilinear  path  steplength  algorithm  for  use  in  minimization 
methods  was  developed.  In  non-convex  regions  the  path  followed 
is  initially  tangent  to  the  direction  of  steepest  descent,  (or 
some  other  suitable  downhill  direction),  eventually  becoming  tangent 
to  a  direction  of  negative  curvature.  In  strictly  convex  regions 
a  straight  path  is  followed.  Under  fairly  modest  conditions 
we  have  proved  that  our  steplength  algorithm  ensures  convergence 
of  the  iterates  generated  by  the  minimization  algorithm  to  a 
critical  point  at  which  the  Hessian  of  the  objective  function  is 
positive  semidefinite. 
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